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@ Method and apparatus for the addition and removal of digital watermarks in a hierarchical Image 
storage and retrieval system. 

@ An image processing technique is described 
in the context of a hierarchical image storage 
and retrieval system. The method allows for the 
controlled addition and removal of digital 
watermarks from selected image components 
in the hierarchy. The method adds a digital 
watermark in a selected image resolution com- 
ponent and the means to remove it in an ad- 
ditional image component termed the 
watermark removal component The method 
employs the encryption of the watermark re- 
moval component and decryption with a spe- 
cial key, or password during authorized 
retrieval. This technique allows users of a distri- 
buted system the convenience of providing the 
entire image hierarchy on a single storage 
medium permitting images containing water- 
marks to be accessed without restriction for 
browsing and proofing, while the watermark 
removal requires knowledge and us of a con- 
trolled code. 



in 




UJ 



FIG 2 



Jouve, 18, rue Saint-Denis, 75001 PARIS 



1 



EP0 651 554 A1 



2 



Cross-reference to Related Application : 

The present application is related to U.S. Patent 
Application Serial No. 08/026,726, entitled "Method 
and Apparatus for Controlling Access to Selected Im- 
age Components In An Image Storage and Retrieval 
System" filed March 5 9 1993, by P. W. Melnychuck 
and assigned to Kodak, the assignee of the present 
application. 

Technical field Of The Invention 

The present invention is related to the field of dig- 
ital image processing and more particularly to meth- 
ods and associated apparatuses for adding and re- 
moving a digital watermark to and from a selected im- 
age resolution and the preventing of unauthorized 
use of associated higher resolution digital image com- 
ponents. 

Background Of The Invention 

A number of hierarchical techniques for image 
coding have been described in the open technical lit- 
erature and in various patents. Of particular rele- 
vance to the present invention are the following pub- 
lications: 

P. J. Burt and E. H. Adelson, The Laplacian Pyr- 
amid As A Compact Code," IEEE Trans. Comm., 
COM-31, 532-540 (1983). 

J. Seberry and J. Pieprzyk, "CRYPTOGRAPHY: 
An introduction to Computer Security" Prentice Hall, 
1988 and the following patents: 

U.S. Pat No, 4,969,204 entitled "Hybrid Residual- 
Based Hierarchical Storage And Display Method For 
High Resolution Digital Images In A Multiuse Environ- 
ment," by Paul W. Melnychuck and Paul W. Jones, 
1990. 

U.S. Pat No, 5,048,111 entitled "Hybrid Subband- 
Based Hierarchical Storage And Display Method For 
High Resolution Digital Images In A Multiuse Environ- 
ment," by Paul W. Jones and Paul W. Melnychuck, 
1991. 

The publication by Burt, et al. teaches an encod- 
ing method for images termed the Laplacian pyramid, 
the Burt pyramid, or the residual pyramid. In this tech- 
nique, the original image is lowpass filtered, and this 
lowpass image is subsampled to take advantage of its 
reduced bandwidth to provide an image of reduced di- 
mension. This process of lowpass filtering and sub- 
sampling is repeated three times to generate a hier- 
archical structure, or pyramid of images of succes- 
sively smaller dimensions. The total number of reso- 
lution levels are created depending on the applica- 
tion. Each lowpass image in this pyramid is then ex- 
panded to the dimensions of the next higher level by 
upsampling (inserting zeros) and filtering to form a- 
prediction image for that level. This prediction image 



is subtracted from its corresponding lowpass image 
in a subtracter to generate difference, or residual, im- 
ages. The residual images corresponding to the lev- 
els of the lowpass pyramid form another pyramid 
5 which is termed the Laplacian, Burt, or residual pyra- 
mid. This technique is motivated by the fact that the 
residual images have a reduced variance and entropy 
compared to the original or lowpass images and may 
be quantized and entropy encoded to provide effi- 
10 cient storage of the data. Reconstruction is per- 
formed by interpolating the decoded lowpass image 
at the bottom of the lowpass pyramid and adding in 
the corresponding decoded residual to generate the 
next level in the lowpass pyramid. This process is it- 
15 erated until the original image size is reached. A pro- 
gressive improvement in reconstructed image quality 
and resolution can thus be obtained by displaying the 
reconstructed lowpass filtered image at each level of 
the pyramid. Note that errors introduced in the encod- 
20 ing process are propagated from one level to the next 
higher level in the decoding process. 

The patent to Melnychuck and Jones (U. S. Pat. 
No. 4,969,204) teaches a modification of the Burt pyr- 
amid scheme by extending the lowpass pyramid 
25 structure to include one or more lowpass filtered im- 
ages of successively smaller dimensions beyond the 
set described by Burt, et al. The advancement in the 
method of Melnychuck and Jones is that the residual 
pyramid is not extended to include these correspond- 
ing extended smaller dimensions. Hence, the Melny- 
chuck and Jones pyramid contains the Burt pyramid 
plus additional lowpass filtered images of smaller di- 
mensions. In a hierarchical image storage and retriev- 
al system, the additional lowpass filtered Images of 
smaller dimension can be retrieved directly, without 
interpolation and addition of residual components. In 
the context of the present invention, the Melnychuck 
and Jones pyramid provides for low resolution images 
that can be used for browsing or proofing. The use of 
these additional low resolution images for browsing 
and proofing means that the customer may use a sim- 
ple retrieval mechanism and need not possess a more 
complex and hence, more expensive retrieval device 
that would be used to decode the higher resolution 
components of the pyramid. Of course, higher reso- 
lution images requiring interpolation and residual ad- 
dition may be used for browsing and proofing as well. 

A hierarchical image processing method will be 
described for the addition and removal of digital wa- 
termarks in selected image components, and for the 
restriction of selected high resolution image compo- 
nents from unauthorized use. An image hierarchy is 
constructed in the context of a multi-resolution envir- 
onment whereby the user has the option of selecting 
the type of display medium and the desired resolution 
of this display medium. In particular, two types of dis- 
play media are considered: video monitors and color 
hard copies, although photographic, thermal imaging. 
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and other types are also of interest In Fig. 1 a prior 
art technique for decomposing, storing, recomposing, 
and displaying, a digital image using a hierarchical 
process is shown. An original digital image is decom- 
posed to provide image versions at various resolu- s 
tions to allow for the display of an HDTV quality image 
on video, an NTSC quality image with PAL/SECAM 
compatibility on video, one or more sub-NTSC quality 
images on video for overviews and browsing, and a 
very high quality image on color hard copy. Intermedin w 
ate to the decomposition and recompositton steps, 
generally are inserted an encoding step, to compress 
the data for storage which in turn requires a decoding 
step when the data is read from storage. 

15 

Summary Of The Invention 

The present invention places a digital watermark 
in a selected image resolution component and the 
means to remove it in an additional image component 20 
termed a watermark removal component Encryption 
of the watermark removal component is used to pre- 
vent use of the image for the generation of unautho- 
rized high quality color hard copy. A watermark is a 
form of graphic overlay that may contain a copyright 25 
notice or information regarding the restricted use of 
the image. In a distributed image system it is common 
to deliver an image of compromised image quality for 
purposes of browsing or proofing. A compromised 
rendition of the image is commonly distributed to pre- 30 
vent full utility orfulfiiiment of the image without prop- 
er payment for the service that generated the image. 
The term browsing refers to the process of image se- 
lection from a plurality of images based on some 
user-defined criterion. Such is the case when a user as 
may select an image from a catalog of images depict- 
ing a particular object The term proofing refers to the 
process of image selection based on the degree of 
desirability of a given image from a plurality of im- 
ages. Such is the case when a professional portrait 40 
photographer distributes a plurality of images to a 
customer for selection and approval. The terms wa- 
termark, browsing and proofing described herein are 
not limited to the examples described above. 

Upon selection of the desired image by the cus- 45 
tomer, the professional delivers a high quality rendi- 
tion of the image, most often in the form of a high qual- 
ity color hard copy. At ail times the professional pos- 
sesses the sole means of generating the high quality 
hard copy. In a conventional photographic system the so 
means would be the original negatives of the images; 
in a digital hierarchical system according to the pres- 
ent invention, the means are higher resolution resid- 
ual components. 

In a digital imaging system, and in particular one 55 
that includes a hierarchical form of digital storage and 
retrieval, the professional may use a suitable digital 
storage medium such as a CD for the distribution of 



proofs. In an unrestricted environment, the customer 
may choose a desired image resolution from the hi- 
erarchy for the purposes of browsing, proofing, or 
hard copy fulfillment In those instances where it is 
desirable for the professional to deliver the digital 
storage medium containing the entire image hierar- 
chy to the customer; it is also most economical to re- 
cord the entire image hierarchy once onto the digital 
storage medium and avoid having to make a second 
copy containing only low resolution components for 
distribution. However, it is also desirable to restrict 
the use of selected high resolution components for 
the purpose of full image quality fulfillment until pay- 
ment has been received. The professional may 
choose to provide low resolution image components 
for browsing or proofing, while maintaining restriction 
of the higher resolution components. Alternatively, he 
may be required to deliver a proof of high resolution. 
Such is the case when the image content contains in- 
formation of small detail and the rendition of this de- 
tail is subject to approval via the proof. With tradition- 
al photographic prints, the professional may place a 
stamp, or watermark on a strategic location on the 
print, so as to render the print useless from a fulfill- 
ment point of view. Note with digital images that ful- 
fillment may mean high quality video at 
NTSC/PAL/SECAM, HDTV, or hard copy. In the pres- 
ent invention, the professional places a digital rendi- 
tion of the watermark on a selected image compo- 
nent The removal of the watermark is done through 
an additional image component containing the re- 
verse of the watermark. The customer, having pos- 
session of the digital storage medium CD would pos- 
sess the means for generating his own high quality 
hard copy when authorized by the professional. Upon 
payment to the professional, the professional or his 
agent provides to the customer the information nec- 
essary to remove the watermark for full image quality 
fulfillment In the present invention, that information 
would be an authorization code, key, or password that 
would be inputted to the image processing system ac- 
cessing the storage medium, to unlock the restricted 
high resolution components. An advantage of this 
technique is that the customer may possess all infor- 
mation pertinent to generating high quality hard copy 
without the need to physically return to the professio- 
nal for additional image components. 

It may additionally be desirable to use some form 
of hierarchical image representation for the purpose 
of browsing or proofing in a distributed system be- 
cause the hierarchy naturally provides a plurality of 
resolutions, and hence levels of image quality, from 
which to choose the proof image. No additional oper- 
ation of compromising the image is necessary; the 
professional simply chooses at what resolution lev- 
el(s) he wants to restrict access. 

Systems that use a hierarchical structuring of the 
image data have not been employed in the past for 
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distribution purposes because of the lack of means to 
simultaneously provide low resolution components 
for browsing and proofing, while offering restricted 
access to the remaining hierarchical components for 
full quality image copy. Additionally, the means to s 
generate and remove a digital watermark in a hier- 
archical image structure had not been previously con- 
sidered. 

The present invention permits the advantages of 
hierarchical image decomposition to create a series 10 
of residual components, direct retrieval of the addi- 
tional low resolution images according to the Melny- 
chuck and Jones pyramid, the addition and removal 
of a digital watermark in a selected image resolution 
component, and prior art encryption methods applied 15 
to the watermark removal component and the resid- 
uals, to provide for a system of browsing, proofing and 
restriction of the high resolution image components 
suitable in a distributed image system. It is assumed 
that the residual components and the watermark re- 20 
moval component are symbol encoded using the en- 
coder box 20 in Figures 2 and 4 into a binary string 
of 1's and O's either via fixed-length coding techni- 
ques (where a binary code word of a fixed-length is 
assigned to each symbol) or variable-length encoding 25 
techniques such as Huffman coding or arithmetic 
coding. The residual data may also be quantized prior 
to encoding, or it may be encoded in a lossless man- 
ner, i.e., without quantization. Data encryption box 26 
is applied to the watermark removal component and 30 
if desired, also to the encoded quantized (or non- 
quantized) residual data. It is assumed that the en- 
cryption process is reversible. Hence, the decryption 
box 28 provides the exact data prior to data encryp- 
tion. 35 

In one embodiment of the invention a storage me- 
dium is called for having stored therein at least one 
low resolution digital image and at least one high re- 
solution digital image, with said high resolution digital 
image encoded with a watermark that requires an au- 40 
thorization code for removal. 

From the foregoing, it can be seen that it is a pri- 
mary object of the present invention to provide a 
method and associated apparatus for storing and 
controllably retrieving digital images stored in a hier- 45 
archical format on a suitable digital storage distribu- 
tion medium that allows the originator of the distribu- 
tion medium to distribute the medium containing the 
entire image hierarchy and a controllably removable 
watermark for the purpose of retrieving low resolution 50 
images for browsing or proofing without compromis- 
ing the originator's need to withhold the means for 
creating hard copies of the images without the water- 
mark. 

It is another object of the present invention to pro- ss 
vide the means for controllably inserting and remov- 
ing a watermark for a digital image. 

It is another object of the present invention to pro- 



vide the means for compromising a selected image 
component of a hierarchical formatted digital image 
by adding a digital watermark to the selected image 
component, and recording the selected image com- 
ponent containing the watermark as part of the image 
hierarchy on a digital storage distribution medium. 

In association with a digital image, it is another 
object of the present invention to provide a means for 
creating a watermark removal component, and for 
controllably restricting access to the watermark re- 
moval component. 

J t is another object of the present invention to pro- 
vide a means for affixing a watermark to a digital im- 
age and for controllably removing the watermark. 

The above and other objects of the present inven- 
tion will become more apparent when taken in con- 
junction with the following description and drawings 
wherein like characters indicate like parts and which 
drawings form a part of the present description. 

Brief Description Of The Drawings 

Fig. 1 is a block diagram illustrating the prior art 
Melnychuck and Jones hierarchical storage and dis- 
play method. 

Fig. 2 is a functional block diagram illustrating a 
hierarchical image decomposition technique incor- 
porating a watermark insertion into an image compo- 
nent. 

Fig. 3 is a functional block diagram illustrating a 
reconstruction technique for reconstructing the im- 
ages decomposed by the system of Fig. 2. 

Fig. 4 is a functional block diagram of another hi- 
erarchical image decomposition technique incorpor- 
ating a watermark Insertion into an image compo- 
nent. 

Fig. 5 is a functional block diagram illustrating a 
reconstruction technique for reconstructing the im- 
ages decomposed by the system of Fig. 4. 

Detailed Description of the Invention 

In the following description of the preferred em- 
bodiments, it will be assumed that the highest reso- 
lution of the image hierarchy is composed of 3072 x 
2048 pixels and that this resolution is adequate to pro- 
duce photographic quality originals on an appropriate 
digital output device. It is also assumed that a moder- 
ately high resolution level of the hierarchy composed 
of 1536 x 1024 pixels is adequate to generate a high 
quality HDTV display, or a small-sized photographic 
quality print on an appropriate digital output device. It 
is also assumed that the lowest resolution levels of 
1 92 x 128 pixels, 384 x 256 pixels, and 768 x 512 pix- 
els are generated and stored onto a digital storage 
medium such as a CD. These resolution levels are 
provided to give the reader an insight as to the oper- 
ation of one or more embodiments of the invention 
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with the understanding that other resolutions or ar- 
rangements may be chosen to suit specific needs 
without detracting from the teachings of the present 
invention. 

Referring now to Fig. 2, a hierarchical residual de- 5 
composition technique, for decomposing a 16BASE 
original image to form a 16BASE residual, a 4BASE 
residual, a BASE, a BASE/4, and a BASE/16 image, 
incorporating the teachings found substantially in Fig. 
7 of the patent to Melnychuck and Jones (U. S. Pat 10 
No. 4,969,204), in combination with the present in- 
vention is shown. The BASE image is processed in 
box 34 to incorporate a watermark and to provide a 
watermarked BASE image. 

An example of a watermark insertion box 34 is 1 s 
given by the watermark insertion unit 22 whereby a 
watermark image W is combined with the input image 
/ to create a watermarked image i w . In this example, 
it is assumed that the input image / and the watermark 
image Ware of the same size and the same bit-depth. 20 
For example, if the input image / is an 8-bit image rep- 
resenting the luminance component of a color image, 
the watermark image W would also be an 8-bit image. 
Similarly, the watermarked image l w would have the 
same size and each pixel value would be represented 25 
with 8 bits. An example of a watermark insertion unit 
22 is one where the input image / and the watermark 
image W are combined according to the following 
equation to create the watermarked image l w 

WJ) = WJ> ♦ a W(ij) 30 

Where (ij) denotes the two-dimensional location 
of the pixels in the image and the operation is per- 
formed for all the pixels in the input image. The wa- 
termark image Wis prepared by the originator of the 
storage medium and may contain the logo of the orig- 35 
inator or any other pattern that the originator may 
wish to use as a watermark. The parameter a, which 
can be either positive or negative, controls the water- 
mark contrast and is also selected by the originator 
and can vary from one image to another. Larger mag- 40 
nitudes of a would, in general, create a higher con- 
trast watermark. Also, to guarantee that the water- 
marked image l w has the same bit-depth as the input 
image /, the watermarked image l w is clipped to the 
same range as the input image. For example, for an 45 
8-bit image with pixel values in the range of 0 to 255, 
for every pixel location (y), the value of f^ij) is dip- 
ped to 255 if the result of the above equation exceeds 
255 and is set to zero if that result is less than zero. 
It should be noted that this example illustrates only 50 
one method of implementing the watermark insertion 
box 34 and the originator of the storage medium may 
incorporate any other method to generate a water- 
mark that creates the desired effect of inhibiting the 
use of the image. 55 

The BASE/16, BASE/4, and watermarked BASE 
images are stored on the digital storage medium 10 
in direct (unencrypted) form. The BASE image, which 



in this case serves as the watermark removal record, 
is encrypted in the data encryption unit 26. The data 
encryption unit 26 consists of either a private-key 
data encryption algorithm (also referred to as sym- 
metric data encryption algorithm) or a public-key data 
encryption algorithm (also referred to as asymmetric 
data encryption algorithm) both of which have been 
explained in the prior art and in the reference book by 
Seberry and Pieprzyk cited before. Examples of pri- 
vate-key encryption algorithms that can be used in 
the data encryption unit 26 are either block ciphers 
such as the Data Encryption Standard (DES) which 
uses a 56-bit key and operates on blocks of data of 
length 64 bits at a time, or a stream cipher algorithm 
such as RC-4, a commercially available encryption 
software that uses a 40-bit key component The en- 
crypted BASE image is also stored on the storage 
medium 10. The 4BASE and 16BASE residual com- 
ponents are also stored on the digital storage medium 
1 0 either in direct (unencrypted) form or in encrypted 
form depending on the level of security desired by the 
application. In the case that the encryption of any or 
all of the residual data are needed, either the same 
key used in encrypting the BASE image is used or a 
separate key is used. The use of multiple encryption 
keys provides the originator of the storage medium 
with more flexibility in controlling the access to the 
various resolutions of the image hierarchy. 

For browsing or proofing, a procedure illustrated 
by Fig. 3 is employed. A user retrieves the BASE/16, 
BASE/4, or watermarked BASE image directly with- 
out decryption from the digital storage medium 10. 
Upon authorization, the user inputs a decryption 
key(s) to the data decryption unit 28 to allow the de- 
cryption of the original BASE image (and the resid- 
uals) to be performed. An example of a data decryp- 
tion unit 28 is a software implementation of a decryp- 
tion algorithm corresponding to the reverse operation 
of the encryption algorithm employed in the data en- 
cryption unit 26. One example of a set of encryp- 
tion/decryption algorithms is the Data Encryption 
Standard (DES) which has been explained in full de- 
tail in the reference book by Seberry et at mentioned 
before. Note that the decryption key(s) must be pro- 
vided by the originator of the storage medium. Upon 
the decryption of the BASE image and the residual 
components, these components can be used to ar- 
rive at full image quality fulfillment 

In a second embodiment, illustrated in Fig. 4, the 
16BASE image is decomposed by decomposition ap- 
paratus 101 into a residual pyramid consisting of the 
1 6BASE, 4BASE, and BASE. The BASE image is fur- 
ther decomposed to create the BASE/4 and BASE/16 
images, through low pass filtering and subsampling. 
BASE /4 and BASE/1 6 are not part of the residual pyr- 
amid and hence they are available directly for display 
on a monitor. 

A watermark, as described in the previous em- 
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bodiment in Fig. 2, is inserted in the BASE image in 
box 34 to arrive at a watermarked BASE image. This 
watermarked BASE image is then interpolated to the 
size of the 4BASE image using linear interpolation as 
indicated by the interpolator box 24. A difference is 5 
formed in subtracter 32 between the original 4BASE 
image and the interpolated watermarked BASE im- 
age to form a modified 4BASE residual that serves as 
the watermark removal record. The difference in this 
embodiment versus the first embodiment is that the 10 
watermark removal record is the modified 4BASE re- 
sidual instead of the BASE image. This modified 
4BASE residual is encrypted using the data encryp- 
tion unit 26 as described before and is then stored on 
the storage media 10 along with the BASE/16, 15 
BASE/4, and watermarked BASE image in direct (un- 
encrypted) form. Finally, the 16BASE residual data is 
stored on the digital storage medium either in direct 
or encrypted form depending on the application. 

For browsing or proofing, the system of Fig. 5 is 20 
employed. The user retrieves the BASE/16, BASE/4, 
or watermarked BASE image directly without decryp- 
tion from the digital storage medium 10. Upon author- 
ization, the user inputs the decryption key to the data 
decryption unit 28 to allow the decryption to be per- 25 
formed to generate the modified 4BASE residual. The 
watermarked BASE image is interpolated using linear 
interpolation and is added to the decrypted modified 
4BASE residual in the reconstruction apparatus 210 
to recover the original 4BASE image. If the residuals 30 
have not been quantized, the 4BASE image can be 
exactly recovered. In the case where the residuals 
have been quantized, some discrepancy between the 
original 4BASE image and the 4BASE image recov- 
ered according to the above scheme would exist The 35 
degree of this discrepancy would depend on the 
coarseness of the quantizer employed in the quanti- 
zation of the residual components. Note that the de- 
cryption key must be provided by the originator of the 
storage medium. 40 

It is to be understood that in some instances it 
may be desirable to place a watermark upon the low 
resolution images to control their access. 

While there has been shown what are considered 
to be the preferred embodiments of the invention, it 45 
will be manifest that many changes and modifications 
may be made therein without departing from the es- 
sential spirit of the invention. It is intended, therefore, 
in the annexed claims, to cover all such changes and 
modifications as may fall within the scope of the in- so 
vention. 
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Claims 

1. A storage medium having stored therein at 
least one low resolution digital image and at least one 
high resolution digital image, with said high resolution 
digital image encoded with a watermark that requires 
an authorization code for removal. 

2. The storage medium according to claim 1 and 
further having stored thereon at least one additional 
high resolution digital image that is not encoded with 
a watermark and is accessed with the authorization 
code in place of the high resolution digital image en- 
coded with the watermark. 

3. A storage medium having stored therein at 
least one low resolution digital image and at least one 
high resolution digital image in the form of a BASE im- 
age, residual image components and a watermark 
component, with said low resolution digital image, 
said BASE image or said high resolution image 
formed by the combination of the BASE image with 
said residual image components and a watermark 
component being accessible without an authorization 
code. 

4. The storage medium of claim 3 in combination 
with an authorization code to remove the watermark 
component from an accessed high resolution image. 

5. A system for controlling the uncompromised 
use of a high resolution digital image stored on a stor- 
age medium as BASE and residual components, 
comprising: 

means for encrypting the residual components 
stored on said storage medium using a watermark 
code; 

means for accessing the BASE and encrypted 
residual components; 

means for combining the accessed BASE and 
residual components to reconstruct the high resolu- 
tion digital image with the watermark code; and 

means for authorizing the removal of the wa- 
termark code. 

& A system for controlling the uncompromised 
use of a high resolution digital image comprising: 

means for forming a hierarchy of lower resolu- 
tion digital images from the high resolution digital im- 
age; 
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means for forming residua! images that are a 
function of differences between adjacent images in 
the hierarchy of lower resolution digital images; 

means for encrypting at least one of the 
formed residual images with a watermark code; 5 

storage means for storing the formed hierarchy 
of lower resolution images and the at least one en- 
crypted residual image; 

means for reconstructing high resolution im- 
ages by accessing and combining a lower resolution w 
image with a residual image; 

means for displaying of the at least one en- 
crypted residual image with the watermark; and 

means for controllably removing the water- 
mark code to permit an uncompromised use of the is 
high resolution digital image. 

7. A recording medium having stored thereon a 
plurality of digital images with each of the digital im- 
ages being comprised of a low resolution digital im- 
age component and at least one residual digital image 20 
component which is combinable with the low resolu- 
tion digital image component to form a higher resolu- 
tion digital image incorporating a watermark which is 
removable with an authorization code. 

& A method for controlling the use of a digital im- 25 
age stored on a storage medium in a hierarchical form 
comprised of a BASE image and at least one residual 
image component, comprising the steps of: 

a) associating a watermark with said at least one 
residual image component; 30 

b) permitting access to the BASE image for low 
resolution viewing of the digital image; 

c) combining the BASE image with the at feast 
one residual image component and an associat- 
ed watermark to form the digital image for view- 35 
ing, printing and/or storing; and 

d) controllably providing a watermark removal 
code to remove the watermark from the formed 
digital image of step c. 

9) A storage medium having stored thereon at 40 
least one digital image encoded with a watermark that 
requires an authorization code for removal. 



7 



EP0 651 554 A1 



I 



L 



HARDCOPY 




HDTV 


OUTPUT 




VIDEO 


PRINTER 




DISPLAY 



DIGITAL IMAGE 



I 



HIERARCHICAL 
DECOMPOSITION 



I 



ENCODER 



STORAGE 
MEDIUM 



I 



DECODER 



I 



HIERARCHICAL 
RECOMPOSITION 



J 



I 



1 



1l 



NTSC 




SUB-NTSC 


VIDEO 




VIDEO 


DISPLAY 




DISPLAY 



FIG. I 
(prior art) 



8 



EP0 651 554 A1 



i 



16 BASE 
ORIGINAL 



DECOMPOSITION 
APPARATUS 101 



16 BASE 
RESIDUAL^ 



20 



26 



ENCODER 



DATA 
ENCRYPTION 
UNIT 



ENCRYPTED 



DECOMPOSITION 
APPARATUS 101 



4 BASE 
RESIDUAL 



20 



ENCODER 



DATA 
ENCRYPTION 
UNIT 



I6BASE 
RESIDUAL 

,26 

ENCRYPTED 



BASE IMAGE 



20 



ENCODER 



DATA 
ENCRYPTION 
UNIT 



4 BASE 
RESIDUAL 

,26 

ENCRYPTED 



BASE 
IMAGE 



i 



LOWPASS 
FILTER 



I 



I 
I 
l 



22 



WATERMARK 
INSERTION 
UNIT 



i WATERMARKED 
I BASE IMAGE l W 
H 



3 WATERMARK V 
w * IMAGE | 



34 



SUBSAMPLE 




BASE/4 IMAGE 



LOWPASS 
FILTER 



I 



SUBSAMPLE 
1 



BASE/16 IMAGE 



FIG. 2 



9 



EP 0 651 554 A1 



ENCRYPTED 
4 BASE 
IMAGE 



WATERMARKED 
BASE IMAGE 




BASE/4 
IMAGE 




BASE/16 
IMAGE 


I 




\ 


DISPLAY 


DISPLAY 


DISPLAY 



DATA 
DECRYPTION UNIT 



DECODER 



BASE 
IMAGE 



4 BASE 
RECONSTRUCTION } RESIDUAL 
APPARATUS 201 ~ 



RECOMPOSED 
4 BASE 
IMAGE 



30 



DISPLAY 



28 



DECODER 





DATA 
DECRYPTION 
UNIT 


ENCRYPTED 




^4 BASE 
RESIDUAL 



DISPLAY 



28 



APPARATUS 201 



IO OM9C 

RESIDUAL 


r 

DECODER 





DATA 
DECRYPTION 
UNIT 



ENCRYPTED 



16 BASE 
RESIDUAL 



DISPLAY 



FIG 3 



10 



EP 0 651 554 A1 



16 BASE 

ORIGINAL 



20 



26 



DECOMPOSITION 
APPARATUS 101 





/ 




DATA 
ENCRYPTION 
UNIT 




16 BASE _ 


ENCODER 


ENCODER 


ENCRYPTED 


RESIDUAL 


16 BASE 
RESIDUAL 


16 BASE 
RESIDUAL 



DECOMPOSITION 
APPARATUS 101 



4 BASE 



IMAGE 



BASE 
IMAGE 




MODIFIED 
4 BASE, 



20 




22 



RESIDUAL 

INTERPOLATED 
WATERMARKED 
BASE IMAGE 



I 

4-»- 



WATERMARK 
INSERTION 
UNIT 

" | WATERMARK 
w 1 IMAGE 



INTERPOLATOR -24 

WATERMARKED 
BASE IMAGE 



LOWPASS 
FILTER 



-^34 



ENCRYPTED 
MODIFIED 
4 BASE 
RESIDUAL 



lf) /CDDISC\ 

»-0 



SUB SAMPLE 



BASE/4 IMAGE 



LOWPASS 
FILTER 



I 



SUB SAMPLE 



BASE/16 IMAGE 



FIG. 4 



11 



EP 0 651 554 A1 




WATERMARKED 
BASE IMAGE 



I 



BASE/16 
IMAGE 



DISPLAY 



DISPLAY 



DISPLAY 



L 



RECONSTRUCTION 
APPARATUS 201 



muuiricu 

4 BASE 


- / . 

DECODER 


RESIDUAL 



RECOMPOSED 
4 BASE 
IMAGE 



2B 

z 



DATA 
DECRYPTION 
UNIT 



ENCRYPTED 
. MODIFIED 



4 BASE 
RESIDUAL 



DISPLAY 



RECONSTRUCTION 
APPARATUS 201 



16 BASE 



RESIDUAL 



30 



28 



DECODER 



RECOMPOSED 
16 BASE 
IMAGE 



DATA 
DECRYPTION 
UNIT 



ENCRYPTED 



16 BA$E 
RESIDUAL 



DISPLAY 



y 

FIG. 5 



12 



EP0 651 554 A1 



European Potest 



EUROPEAN SEARCH REPORT 



EP 94 42 0293 



Category 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Citation «f 



wrtfc 



US-A-4 696 20* (MELNYCHUCK ET AL) 

* the whole document * 

ELECTRONICS AND COMMUNICATIONS IN JAPAN, 
vol.73, no. 5, May 1990, NEW YORK, US; 
pages 22 - 33 

N.KOMATSU ET AL "A Proposal on Digital 
Watermark In Document Image Communication 
and Its Application to Realizing a Digital 
Signature* 

* figures 1-5 * 

* page 22, left column, line 1 - page 27, 
left column, line 23 « 

EP-A-0 614 308 (EASTMAN KODAK) 

* abstract; figure 2 * 

* column 4, line 52 - column 5, line 4 * 



to daim 



1-9 



1-9 



■ *«wn op fer ail< 



THE HAGUE 



PohofoMjloM oflk—di 

30 January 1995 



CLASSIFICATION OF THE 
APPLICATION (lalCU) 



H04N1/21 
G06F1/00 



H04N 
G06F 



Powell , 0 



CATEGORY OF CITED DOCUMENTS 

X : ptfttatvly relevant If takm «Iom 
V : ggg toif ntonmx U eonUMri wttfa maathm 
ncaaol of tbo I 

AH 

O j ttNMvftm ilsda 
Pel 



T ; ttaorjr or oriftdple ntoMng tfe« far 
E : «ifl* petmx tec— rt, bat wUtefc 

tftor dwrttinfet* 
D:«ocMMrtdMte« 
Li iiMtdul 



■raws 



13 



